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Abstract 

In game theory, an Evolutionarily Stable Set (ES set) is a set of 
Nash Equilibrium (NE) strategies that give the same payoffs. Similar 
to an Evolutionarily Stable Strategy (ES strategy), an ES set is also a 
strict NE. This work investigates the evolutionary stability of classical 
and quantum strategies in the quantum penny flip games. In partic- 
ular, we developed an evolutionary game theory model to conduct a 
series of simulations where a population of mixed classical strategies 
from the ES set of the game were invaded by quantum strategies. We 
found that when only one of the two players' mixed classical strategies 
were invaded, the results were different. In one case, due to the in- 
terference phenomenon of superposition, quantum strategies provided 
more payoff, hence successfully replaced the mixed classical strategies 
in the ES set. In the other case, the mixed classical strategies were 
able to sustain the invasion of quantum strategies and remained in the 
ES set. Moreover, when both players' mixed classical strategies were 
invaded by quantum strategies, a new quantum ES set emerged. The 
strategies in the quantum ES set give both players payoff 0, which is 
the same as the payoff of the strategies in the mixed classical ES set 
of this game. 

1 Introduction 

Game theory applies probability theory to address uncertainty in the deci- 
sion making process. In the classical world, players' decisions are stored in 
classical channels and the information is processed based on classical me- 
chanics. With the advent of quantum computing, it is natural to ask if 
quantum mechanics would change the dynamics of game playing. To put it 
another way, if the decisions are stored in quantum channels, communicated 
and processed under quantum mechanics, would a game still produce the 
same result? 



In 1999, Meyer [10] explored quantum superposition to play a two-person 
zero-sum game: the quantum penny flip game. He reported that due to the 
interference phenomenon of superposition, the game that originally has a 
Nash Equilibrium (NE) strategy that gives both players equal probability 
to win in the classical world has become biased: when the first player is 
allowed to use quantum strategies, this player always outperforms the other 
player who uses mixed classical strategies. Around the same time, Eisert and 
colleagues [2] investigated quantum entanglement in playing the Prisoners' 
Dilemma (PD) game. They quantized the game by first entangling two 
qubits using a unitary operator (explained in Section [3]) that was known to 
both players. After that, the two qubits were distributed to the two players 
to encode their decisions. Once returned, the two qubits were disentangled 
using another unitary operator. Finally, the two qubits were measured and 
the results were used to calculate their payoffs. By leveraging quantum 
entanglement, the authors reported that the dilemma of the game did not 
exist any more. Moreover, the mutual- defection classical NE strategy was 
replaced by a quantum NE strategies that gave both players higher payoffs. 
Since then, various new development on quantum games have been reported, 
such as the quantization^] of the Battle of the Sexes game jS], the Hawk-Dove 
game [E] , the Monty Hall problem |3j and evolutionary quantum games [7] . 

Although with higher payoffs, are those quantum NE strategies evolu- 
tionarily stable? An NE strategy is evolutionarily stable if a population 
of players have adapted the strategy, natural selection alone is sufficient to 
prevent it from being invaded by any mutant strategies [9] (more details 
in Section [2]). Iqbal and Toor [6] were the first to investigate this problem 
using the PD game. They reported that when invaded by a group of one- 
parameter quantum strategies, the classical mutual-defection NE strategy 
remained evolutionarily stable. However, when invaded by a group of two- 
parameter quantum strategies, they were no longer stable and were replaced 
by a higher-payoff quantum mutual- collaboration strategy. Moreover, the 
new dominating mutual-collaboration quantum strategy was evolutionarily 
stable against the invasion of any two-parameter quantum strategies. 

The objective of this study is to investigate the evolutionary stability 
of classical and quantum strategies in the quantum penny flip game, which 
was introduced by Meyer [TU] (see Section [4] for more details). In addition 
to deriving analytical results, we are also interested in understanding the 

1 Quantization here refers to "deriving a quantum version of a classical algorithm", 
which is different from "the process of converting analog to digital signals" that is more 
popular in the wider scientific community. 
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dynamics of strategy changes during the game. Evolutionary game theory 
(EGT) models [§] are ideal vehicles to provide such insight. In an EGT 
model, there is a strategy replication rule, in addition to the game contest- 
ing rules. The extra rule specifies how fitter strategies are multiplied and 
how less fit strategies are culled out of a population. Games are played 
repeatedly for many generations for the dynamics of the strategy changes 
in the population to emerge. EGT models have been successfully applied to 
analyze strategy changes, the success/failure of a strategy and the existence 
of equilibrium strategies in a game [3]. 

This research developed an EGT model to study the quantum penny 
flip game. In particular, we used the model to conduct a series of computer 
simulations where a population of mixed classical strategies from the Evolu- 
tionary Stable Set (ES set) of the game were invaded by quantum strategies. 
The game has two players where one player has single move while the other 
player has two moves. If both players use classical strategies, the two-move 
player does not have advantage over the one-move player; the game always 
ends with an equilibrium state where both players receive the same payoff 
of 0. However, if the two-move player is allowed to use quantum strategies 
while the one- move player is not, this is no longer true. Our simulation 
results show the following interesting phenomena: 

• The two-move player's mixed classical strategies are not evolutionarily 
stable under the invasion of pure quantum strategies, where one par- 
ticular kind of quantum strategy gives a higher payoff when playing 
against the one-move player's mixed classical strategies. This result 
is identical to that reported in |10j . Moreover, once they become the 
dominating strategies in the population, this kind of quantum strate- 
gies are evolutionarily stable against the invasion of any classical and 
quantum strategies and make the two-move player win the game with 
certainty. 

• The one-move player's mixed classical strategies are evolutionarily sta- 
ble under the invasion of pure quantum strategies when playing against 
the two-move player's mixed classical strategies. In other words, pure 
quantum strategies do not provide advantage over classical strategies 
for the one- move player in this game. 

• When both players' mixed classical strategies were invaded by quan- 
tum strategies, a new quantum ES set emerged. The strategies in the 
quantum ES set give both players payoff 0, which is the same as the 
payoff of the strategies in the mixed classical ES set of this game. 
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In addition to being the first to investigate evolutionary stability of clas- 
sical and quantum strategies in the quantum penny flip game, this work also 
made the following novel contributions: 

• It proposed and implemented an EGT model to conduct systematic 
study of Evolutionarily Stable (ES) strategies for quantum games in 
general and the quantum penny flip game in particular (see Section 

• It analyzed and interpreted the EGT model simulation results for 3 



variations of the quantum penny flip games (see Section 5.1 and Sec- 
tion 



5.2). 



• It identified a quantum ES set where all strategies in the set give both 
players payoff 0, which is the same as the payoff of the strategies in 
the mixed classical ES set of this game. 

This work is significant in the following ways: 

• It demonstrates how evolutionary game theory models can advance 
our understanding of quantum game theory. 

• It demonstrates that even the simple quantum penny flip game has a 
rich problem landscape, which could be an inspiration for the investi- 
gation of other quantum games using a similar approach. 

The paper is organized as follows. Section [2] first explains NE and ES 
strategies in two-player symmetric and asymmetric games. After that, their 
extension to the Evolutionarily Stable Set (ES set) under evolutionary game 
theory models are described. Section [3] gives a brief introduction to quan- 
tum information processing. In Section |4j the quantum penny flip game is 
described and its mixed classical NE strategies that form the ES set of the 
game are analyzed. Section [5] presents our work investigating classical and 
quantum ES sets in 3 variations of the game. Finally, Section [6] concludes 
the paper and outlines our future work. 



2 Nash Equilibrium &; Evolutionary Stability 

In classical game theory, a profile of all players' strategies is a Nash Equi- 
librium (NE) if none of the players can do better by changing his or her 
strategy unilaterally. It is assumed that all players have knowledge of the 



4 



other players' strategies and are allowed to use that information to maximize 
their own payoffs. 

In mathematical terms, a strategy profile x* = {x*} is a NE under the 
following condition: 

Mi <E N,Xi G Si,Xi ^ x* : fi(x*,x*_A > fi(xi,x*_A (1) 

where strategy of player i within his strategy set Si and ft is his payoff 

function. Also, x*_ i is a strategy profile of all players except for player i. 

Evolutionary stability is a refinement of NE. A NE strategy is an Evo- 
lutionarily Stable (ES) strategy if when it is fixed in a population, no al- 
ternative (mutant) strategy can invade the population successfully. In a 
two-player symmetric game, where both players use identical strategies set 
and have identical payoff function, x is an ES strategy under the following 
conditions: 

f(x,x)> f(y,x), (2) 
if f(x,x) = f{y,x) then f{x,y) > f{y,y) (3) 

where y is any alternative (mutant) strategy and / is the payoff function of 
the game. 

In two-player asymmetric games, one population is not sufficient to 
model the game. Hofbauer and Sigmund [4] modeled the game using two 
populations, where each population contains strategies that belong to one of 
the two players to play against each other under two different payoff func- 
tions. A pair of strategies (p, q) is an NE pair if the following condition 
holds: 

M(x,y) e S A x S B : 

Ia(p, q) > Ia(x, q) and f B (p, q) > f B (p, y); (4) 

where Sa and S B are the strategy sets for populations A and B respectively; 
/a and f B are the payoff functions for strategies in population A and B 
respectively. 

Moreover, (p, q) is an ES pair under the following condition: 

y x ^p,y^q: 

fA(p,q) > fA(x,q) and f B (p,q) > f B (p,y); (5) 

Equation [5] is the definition of strict NE. Hence, in two-player asymmetric 
games, only strict NE strategies are ES strategies. 
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Thomas |17j extended the concept of ES strategies to Evolutionarily 
Stable Set (ES set) for games which do not have an ES strategy but have a 
continuum of NE strategies that give the same payoffs. These NE strategies 
form an ES set, which is a strict NE like an ES strategy for these games. 

Thomas showed that under the standard evolution dynamics, if an EGT 
model starts with a population that is in the neighborhood of an ES set, 
the population would converge towards some of the strategies of the ES 
set. In Section [5j we will seed our EGT model with strategies from the 
mixed classical ES set and then examine if the population converges to any 
strategies of an quantum ES set, if it exists. 

In order to apply the concept of ES sets to quantum strategies, we have to 
address one important question: "does evolution take place in the quantum 
realm?" More precisely, could selection operate on superpositions without 
measurement? Could quantum information be mutated and inherited from 
one generation to another? According to Quantum Darwinism |18| . the an- 
swer is "yes" : quantum states are selected against each other in favor of a 
stable pointer state. We will give a brief introduction to quantum informa- 
tion processing in the following section to pave the way for our investigation 
of quantum ES sets. 

3 Quantum Information Processing 

In classical computing, information is stored in binary or 1. By contrast, 
quantum information resides in superpositions of and 1. Quantum infor- 
mation processing takes place on these superpositions simultaneously using 
unitary operators (explained later). During the process, however, no prob- 
ing is allowed to know the intermediate states of the information. This is 
because once measured, quantum information collapses into classical value 
of or 1. Meanwhile, all superposition information is destroyed. Therefore, 
measurement is only performed at the end of the information processing to 
obtain the final result. 

A quantum bit (qubit) is the counter-part of a classical bit. Using Dirac 
notation p], a qubit is represented as a linear combination of basis states 



The state of a qubit, hence, is a vector in a two-dimensional complex vector 
space. The states |0) and |1) are computational basis states, which form an 
orthonormal basis for this vector space. 



|0) and |1): 




(6) 
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When a qubit is measured, it collapses to |0) with probability acQ or to 
|1) with probability bb. Hence, aa + bb = 1. 

In addition to the above vector representation, the state of a qubit can 
also be formulated using the density matrix |15j : 



p = |^)(VI 

For example, the associated density matrix for 



|0> is: 



[1 0] 



1 





(7) 



(8) 



Similarly, the associated density matrix for = ^(|0) + |1)) is: 



l 

V2 



1 1 

s/2 



1 I 

1 1 

2 2 



(9) 



A mixed quantum state is a linear combination of pure state \tpi), each with 
probability pf. 

For example, the associated density matrix for a mixed quantum state being 
in |0) and |1) with equal probability is: 



[1 0]+^ i [o i 



o i 



(11) 



The transformation of a quantum state to another is through a unitary 
operator U, where C/f71j= /: 



Pi+l 



(12) 



A quantum state can also be transformed by a mix of j unitary operators, 
each with probability pj : 



Pi 



+1 = J2 P 3 U 3P iU j 



(13) 



The trace of a density matrix, p, is always 1. The top left corner value, 
p(l,l), gives the probability of the qubit being measured as |0). The bottom 



The a defines complex conjugate of a. 
3 The f notion defines Hermitian conjugate. 
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right corner value, p(2,2), gives the probability of the qubit being measured 
as |1). 

For example, the measurement of p in Equation Q produces classical 
value 0, while the measurement of p in Equations ^ & (11) produces clas- 
sical value of and 1 with equal probability. From the measurement point 
of view, the two states in Equations Q Sz (11) are identical. However, if p 
is an intermediate state, not to be measured, farther transformation of the 
two p may lead to different states, hence different final measurements. The 
p in Equation ([9| describes a quantum state as a linear superposition, where 
quantum interference may cancel or enhance the probability of the measure- 
ment results. By contrast, the p in Equation (11) describes a classical state, 
where the interference phenomenon does no exist. 

The above quantum states and unitary operations can be extended to 
multiple qubits using the tensor operator ®. However, since the quantum 
penny flip game only operates on one qubit, we will not discuss multiple 
qubits operations. We refer interested readers to 



4 Quantum Penny Flip Games 



Meyer |lUj quantized the classical Matching Pennies game in the following 
ways. Initially, the penny is placed in a closed box head- up. Next, the first 
player (Q) is allowed to flip the penny. Next, the second player (Picard) is 
allowed to do the same. After that, Q has a second turn to flip the penny 
if he wishes. Since the penny flipping is carried out in a closed box, the 
intermediate state of the penny is unknown. It is only after Q has made his 
second move, the box is opened and the state of the penny is measured. If 
the penny is head-up, Q wins. Otherwise, Picard wins. The payoff is 1 for 
the winner and -1 for the loser. This is an asymmetric game, because the 
two players have a different number of moves and their payoff functions are 
different. 

As shown in Section [3j both classical and quantum states can be rep- 
resented in density matrices, while classical and quantum strategies can be 
represented as unitary operators. Using the qubit state |0) to represent 
head- up and 1 1) to represent tail- up of a penny, the initial head- up state of 



the penny is po 



; the classical flip unitary operator is F 



and the classical no-flip unitary operator is N 



If both Q and Picard use classical strategies, this three-move game has no 



S 



pure classical NE strategies. However, it has a continuum of mixed classical 
NE strategies (|, \, *) U (*, \, \): Q flips the penny with probability \ 
in one of his two moves and uses an arbitrary strategy in the other move; 
Picard flips the penny with probability The result is a tie, where both 
players have equal probability to win the game. In other words, Q does not 
have advantage over Picard due to his one extra move. 

To understand this continuum of mixed classical NE strategies, we have 

to explain two observations. First, the state p = 



is a "stuck state" , 



where no classical or quantum strategy can change the state. 

In the classical case, let an arbitrary mixed classical strategy have prob- 
ability p to flip the penny and probability (1 — p) not to flip the penny 
(note that pure strategies are a special case where p = 1 or p = 0). The 

"- 01 

strategy transforms pi = "i i to pi + \ = pFpiF^ + (I — p)N piN^ = 

y 2~J 

In the quantum case, let U = 
turn strategy, which would transform p. 





l 

2J 



be an arbitrary pure quan- 
to p i+ i = UpiW = 



a 


b ' 




■i 

2 


0" 




a 


b ' 




■i 

2 


0" 


b 


—a 







1 

2_ 




b 


—a 







1 

2. 



Second, a mixed classical strategy with p = ^ 
an arbitrary classical state p 



shown below: 



2 r 



1 



(half- half) can transform 
\ 01 



to the stuck state 







as 



~s 




s 


Art = 


■i 

2 


0" 


1 - s 




1 - s 





1 

2. 



Hence, regardless what classical strategy Q uses in his first move, Picard's 
half-half mixed classical strategy will transform the penny to the stuck state. 
Once stuck, Q's second classical strategy can not change the state to a 
different state. The game ends in a tie where both Picard and Q have equal 
probability to win the game. Note that one of Q's two moves has to be 
the half-half mixed strategy. Otherwise, Picard can change his half-half 
mixed strategy to a different strategy to improve his payoff according to the 
non-half-half strategy that Q plays. 



The continuum of NE strategies 



- 1 l *\ 

-2' 2' I 



u (" 



i i- 

' 2' 2- 



is the classical ES 



set for this game. Any classical strategies that is outside this set would give 
one of the two players expected payoff that is less than 0. Under selection 
pressure, this kind of strategies will become extinct in the population. 
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5 Quantum Evolutionarily Stable Sets 



Starting with a population of strategies from the mixed classical ES set, 
we design the following three mutant invasion simulations to investigate the 
existence of quantum ES set in the quantum penny flip games: 

1. Q's mixed classical strategies in the ES set are invaded by pure quan- 
tum strategies to play against Picard's mixed classical strategies in the 
ES set; 

2. Picard's mixed classical strategies in the ES set are invaded by pure 
quantum strategies to play against Q's mixed classical strategies in 
the ES set; 

3. Q's mixed classical strategies in the ES set are invaded by pure quan- 
tum strategies while Picard's mixed classical strategies in the ES set 
are invaded by mixed two quantum strategies to play against each 
other. 

We designed our EGT model based on the evolutionary framework de- 
scribed by Hofbauer and Sigmund [3] to conduct the designed simulations. 
In this model, there are two populations P and K, one for Picard's and 
one for Q's strategies. At each generation, every p G P plays against every 
k G K. At each contest, the payoff of p is the probability of the penny's 
final state (pfi na i) being measured as |1) minus the probability of it being 
measured as |0). By contrast, the payoff of k is the probability of p final be- 
ing measured as |0) minus the probability of it being measured as |1). The 
fitness of p is the average payoff of its contests against all k £ K. Similarly, 
the fitness of k is the average payoff of its contests against all p G P. 

M = Efc£ ^ (P ' ^ , fp(p, k) = P final(l, 1) ~ Pfinal(0, 0) (14) 
f( k ) = ^ pep f*^ > ' ,f K (j>, k) = pf ina l(0, 0) - Pfinal{l, 1) (15) 

We used the following two-parameter unitary operator to represent a quan- 
tum strategy: 

cos6 —e l ^sint 
sinO e^cosO 



u(e,<t>) 



where 6 G [0, f] and 4> £ [0,7r]- The classical flip strategy F is t/(§,7r 



1 

1 



. The classical no-flip strategy iV is U(0, 0) 



1 
1 
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We implemented the EGT model using Holland's Genetic Algorithms 
(GAs) |5J. An individual is a linear chromosome that consists of a number 
of genes. To encode a mixed classical strategy, the gene is the probability 
(pro) to flip the penny. To encode a pure quantum strategy U, the genes 
are the values of 9 and eft. Encoding a mixed-two quantum strategy requires 
5 gene values: the probability of applying the first quantum strategy, the 6 
and 4> of the first quantum strategy and the 9 and <f> of the second quantum 
strategy. 

The half-half mixed strategy can be coded in two different ways: p = 0.5 
or U(j, *), where * is a random value between and tt. This is because when 

Is 



applied to an arbitrary classical state pi 



1 



f7(f ,*) produces the 



same effect as the half- half mixed classical strategy on the diagonal elements: 



(H+i = U(h*) 



s 
1 - s 



When measured, the penny collapses to |0) and |1) with equal probability. 

Note that is not the same as the "stuck state" , since its off-diagonal 
values are not zero. If pj+i is an intermediate state of the penny, quantum 
strategies can change the state although classical strategies cannot. Hence, 
the final measurement of the penny may not be a tie. However, in simula- 
tions 1 &: 2, one of the two players remains using classical strategies, which 
can not modify pi+i- The U(|,*) strategy, therefore, behaves identical to 
the classical half-half mixed strategy. 

The system uses the following standard GA operators: 

• Binary tournament selection: two chromosomes are randomly selected 
from a population and the one with higher fitness is the winner. 

• Gaussian mutation: each gene value of a selected chromosome has a 
specified probability to be mutated by adding random values from a 
Gaussian distribution under a specified standard deviation to produce 
a new offspring. 

• Average crossover: the genes of two selected chromosomes are averaged 
to produce one offspring. 

Figure [5] gives the GA system workflow. Initially, two populations, P and 
K, are seeded with classical strategies from the ES set. The two populations 
of strategies then play against each other and their fitness values are evalu- 



ated according to equations 14 & 15 Based on the fitness, fitter strategies 
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are selected to perform average crossover and Gaussian mutation to produce 
offspring for the next generation. This process of selection-reproduction- 
evaluation is repeated for many generations until the maximum number of 
generation is reached. 



Begin GA 

g:=0 { generation counter } 
Initialize population P(g] 
Initialize population K[g] 
Play each p in P(g] against each k in K(g] 
{compute fitness values} 
While g < max_gen do 
g:=g+l 

While the size of P(g] < pop_size 

Apply Binary Tournament Selection twice to 

chose two winners from P(g-1] 
Perform average crossover on the two winners 

to produce one offspring 
If (rnd(]<mutation_rate] 

Perform Gaussian mutation on the offspring 
Add the offspring to P(g] 
End while 

While the size of K(g] < pop_size 

Apply Binary Tournament Selection twice to 

chose two winners from K(g-1] 
Perform average crossover on the two winners 

to produce one offspring 
If (rnd(]<mutation_rate] 

Perform Gaussian mutation on the offspring 
Add the offspring to K(g] 
End while 

Play each p in P(g] against each k in K(g] 
{compute fitness values} 
End while 
End GA 



Figure 1: The genetic algorithm system work flow. 



Table [T] lists the GA parameters used to run the simulations. The max- 
imum number of generation is 500 for simulations 1 & 2 and 10,000 for 
simulation 3. We present simulation 1 & 2 and their results in the following 
subsection. In Section 5.2 we present and analyze the results of simulation 
3. 



Table 1: GA Parameter values to run the simulations. 



parameter 


value 


parameter 


value 


pop_size 


50 


max_gen 


500/10,000 


mutatioiuate 


20% 


Gaussian mutation std 


0.2 
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5.1 Quantum Strategies Invade One of the Two Players' 
Classical Strategies 

As analyzed in Section |ZJ the game has a mixed classical ES set (|, |, *) U 
(*, \,\)- We encode the half-half strategy as t7(|, *) in the population that 
is to be invaded by quantum strategies, which can be population P or K, 
hence any U(9 ^ *) is a quantum mutant strategy. By contrast, the half- 
half strategy in the population that is not invaded by quantum strategies, 
which can be population P or K, is coded as pro = ^. In this way, the pro 
can be mutated to any probability to adapt to the invasion that is taking 
place in the other population (see the work flow in Figure [5]) . 



5.1.1 Simulation 1: 



In this simulation, K is invaded. The initial K population is therefore seeded 
with each k 6 K having two unitary operators Ul and U2, one for each of 
its two moves, as C/(J,*) and [/(*,*), where * is a value randomly chosen 
such that < 9 < ? and < <j) < it. By contrast, all p £ P are identical 
with pro = 0.5. We made 100 simulation runs and the population average 
fitness with the standard error of the mean (SEM) are given in Figure [2j 



K pop avg fitness •— 

K pop best stgy fitness >- 

P pop avg fitness 

P pop best stgy fitness ■ 



100 200 300 
generation 



180 
160 
140 
120 
100 
80 
60 
40 
20 




^ iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii 



pop avg Ul 8 
K pop avg Ul (|> 
K pop avg U2 9 
K pop avg U2 <|) 
P pop avg pro 



200 300 
generation 



Figure 2: P and K avg population Figure 3: P and K strategies con- 
fitnesses, tents. 

At generation 0, both populations were seeded with mixed classical 
strategies from the ES set, hence all individuals received payoff 0. Once 
some of the U in the K population and some of the pro in the P population 
were mutated, the average fitness of the K population increased very quickly, 
while the average fitness of the P population dropped very quickly. At gen- 
eration 400, the average fitness of both populations converged, where all 
quantum strategies in the K population received fitness close to 1 (0.9999) 



13 



while all mixed classical strategies in the P population received fitness close 
to -1 (-0.9999). 

Figure [3] shows the evolved strategies in both populations. For the K 
population, they are the average 9 and of Ul and U2 with the SEM. For 
the P population, they are the average pro with the SEM. 

At generation 0, the K population consisted of two types of individuals: 
[U(^,*),U(*,*)] and [[/(*,*), U{\, *)], where * is a randomly generated 
value that satisfies the constraints, < 9 < | and < <j> < it. All pro in 
the P population, on the other hand, are 0.5. 

After the evolution started, the average 9 of the two quantum strategies 
Ul and U2 in the K population remained as | while the average (j) of Ul 
stayed around | but with a large standard error. By contrast, the average 
cf) of U2 grew and converged to ir around generation 400. Meanwhile, the 
average pro of the P population fluctuated between 0.4 and 0.6, but did not 
converge to a particular value. 

We analyze the penny states using the evolved strategy [U( \ , *) , pro, U(j,n)]. 

"1 0l 



Given the head-up state, po 







the state of the penny after Q 



applied I7(f,*) is pi = U(3,*)poU( 



4 - 



*)t 



Next, Picard ap- 



plied his mixed classical strategy (pro) to transform the penny state to 

i i- 

2 



P2 = proFpiF^ + (1 — pro)N p\N^ 



2 
1 

L2 



I 
2J 



which is identical to p\. In 



other words, Picard's mixed classical strategy could not change the state of 
the penny. Since all pro values produce the same the pro in P popu- 
lation had a wide range between and 1 with average around 0.4 and 0.6, 
as that shown in Figure [3j Note that unlike the "stuck state" , the phase 
(non-diagonal value) of p2 is not zero. Although classical strategies cannot 
change the state, quantum strategies can modify the phase and through the 
effect of interference (see Section [3]), the state of the penny may change. 

If both players had only one move, the game would have ended and both 
players received the same payoff of 0. However, this is not the case, as 
Q had one more move. He then applied U(j,ir), whose interference effect 

changed the state to: p3 = U , tt) P2U (j , ir)^ = . When measured, 

the penny collapses to |0) with probability 1. The game therefore ended 
with Q receiving payoff 1 and Picard receiving payoff -1. While the extra 
move did not give Q advantage in the classical version of the game, it helped 
him to win in this version of quantum penny flip game with certainty. 



14 



Why did the 
initial head-up state po 



of Ul have such a large standard error? Given the 
the state of penny after applying an 



arbitrary strategy U(9, (f) 

1 0' 
1 



cosO 


-e* 


sinO 


sinO 


e^cosO 


n other words, <b 




-\ i- 




Pi = 


2 2 
1 1 

.2 2_ 


, the 





cosO 
sinO 
cosO 



-e^sinO 
e^cosQ 
sinO 



is: p 1 = U(0,cP) Po U(e,cj ) ) 



sinO 



cost 



cosO 2 
sinOcost 



sinOcosO 
sin9 2 



of £71 has no impact on p%. Since all U(|,*) would produce 
<fi can be any value between and tt. Consequently, they 

average to | with a large standard error as that shown in Figure [3j 

The set of strategy pairs [U(j,*),U(j,tt)] are the winning quantum 
strategies for Q, regardless what mixed classical strategies Picard uses. 
Moreover, any [U(9 / j,*),U(9 ^ \ 7^ n)} would give Q a lower payoff. 
The continuum of [U(j , *), *, £7(f , 7r)] is therefore the ES set for this version 
of the quantum penny flip game. 



5.1.2 Simulation 2: 

In this simulation, P is invaded. The initial P population is therefore seeded 
with each p as the unitary operator ,*), where * is a random value 
between and tt. By contrast, the K population is seeded with each k as 
either \pro\ = 0.5,pro2 = *] or \pro\ = *,pro2 = 0.5], where * is a random 
value between and 1. We made 100 simulation runs and the results are 
presented in Figure [4] and Figure [5] 
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Figure 4: P and K avg population Figure 5: P and K strategies con- 
fitnesses, tents. 

Figure [4] shows that during the entire simulation of 500 generations, the 
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average fitness for both populations stayed close to 0. Figure [5] shows that 
the dominating strategy in the P population converged to U( J , § ) while the 
average pro\ and pro2 in the K population fluctuated between 0.4 and 0.6 
but did not converge to a particular value. 

To analyze the ES set for this version of quantum penny flip game, we 
evaluate the penny states under the three evolved operations [pro\ ,U(|, | ), 

"1 0l 



pro2\. With initial state po 



after Q applied his mixed classical 





strategy pro\ , the state of the penny is p\ = proiFpoF T+(l— pro\)N p^N^ 
pro\ 



1 — pro\ 



Next, Picard's f7(? , | ) would transform the state to p2 



tf(f, 0)^17(1, 0)t = | 



In fact, Picard can use any 



1 2pro\ — 1 

2pro\ — 1 1 

U(j, *) to transform the penny to the same state, since <f> has no impact on 
the state transformation. We have examined the P population and found 
that the <j> values spread between and n, hence averaged to |. With all 
100 simulation runs having average <f> ~ their average is also close to | 
with a small standard error, as that shown in Figure [5] 

Finally, Q applied his second mixed classical strategy pro2 to transform 



1 2pro\ — 1 

2pro\ — 1 1 



the penny to p3 = pro2F p2F^ + {l—pro2)N p2N^ = ^ 

which is identical to p2- Once measured, the penny collapsed to |0) and |1) 
with equal probability, hence both players received expected payoff 0. Since 
neither pro\ nor pro2 has impact on final state, hence the measurement re- 
sult, their values can have a wide range between and 1. Consequently, 
their averages are around 0.4 and 0.6, as that shown in Figure [5j 

This result is similar to the classical version of the game in that no 
matter what mixed classical strategies Q used in his first move, Picard's 
U(j, *) half-half strategy will transform the penny to a stuck state, where 
Q's second classical strategy could not change the state to other state. The 
quantum version of the game therefore has the same ES set as that of the 
classical version of the game: [^, U(j, *), *] U [*, U(j, *), h]. 

Classical strategies are a proper subset of quantum strategies. The ES 
set in the classical version of a game therefore does not need to be the ES 
set in the quantum version of the game. This is the case in simulation 1 
where the mixed classical strategies in the ES set are replaced by quantum 
strategies in that version of quantum game. However, in simulation 2, the 
mixed classical ES set remain as the ES set in this version of quantum game. 
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5.2 Quantum Strategies Invade Both Players' Classical Strate- 
gies 

In this simulation, the mixed classical strategies in the K population are in- 
vaded by pure quantum strategies, while the mixed classical strategies in the 
P population are invaded by mixed-two quantum strategies. We designed 
this simulation as an extension of simulation 1 to explore whether mixed- 
two quantum strategies can provide better payoff than mixed-two classical 
strategies for Picard when playing against Q's pure quantum strategies. 

Similar to simulation 1, the initial K population is seeded with two types 
of individuals: [U(f ,*), U(**)] and [U(**)> U(f, *)]. Unlike simulation 1, 
all p G P in this simulation are [pro = 0.5, ?7(f , n), U(0, 0)], where f (f , n) = 



is the classical flip strategy and U(0, 0) 



is the classical no- 



dip strategy. In this way, the quantum strategies in both populations can 
evolve against each other to improve their payoffs. We made 100 simulation 
runs and the average population fitness with the SEM are presented in Figure 

m 



i 



0.5 

0) 

C 

-p 

■H 



-0 . 5 



-1 

2000 4000 6000 8000 1000C 

generation 

Figure 6: P and K population average fitness. 

Similar to the two previous simulations, both populations had the same 
initial average fitness of 0. However, once evolution started, the average 
fitness of K population increased while the average fitness of P population 
decreased. This trend continued until generation 800 when the trend is 
reversed: the average fitness of P population grew while the average fitness 
of K population shrank. Around generation 4,000, the average fitness of both 
populations converged to 0, indicating all strategies in both populations give 
equal probability to win the game. 

Compared to the two previous simulations, the populations in this sim- 




K pop avg fitness 
""pop best stgy fitness 
P pop avg fitness 
P pop best stgy fitness 
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ulation took 4 times longer to converge. This is because the number of 
strategy parameters in this simulation (Q has 4 parameters while Picard 
has 5 parameters) is larger than that in the two previous simulations (Q 
has 4 or 2 parameters while Picard has 1 or 2 parameters). With a larger 
parameters space, it took evolution longer to find the stable strategies for 
this version of quantum penny flip game. 

To understand the dynamics of strategies changes during the game, we 
examined the 100 evolved quantum strategies in both populations. We 
found that they can be grouped into 4 categories as shown in Table [2] The 
descriptions of these quantum strategies are given in Table [3} Note that 

01 j 02 > 03 are Pauli matrices. We represent o-i as 



1 



which behaves 



identical to 



when applied to an arbitrary state p 



a 

1-s 



-1 




s a 




-1 
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-i 




s a 




-i 


t 


1 — s —a 


1 




a 1 — s 




1 




i 




a 1 — s 




i 




—a s 



Table 2: Four categories of the evolved quantum strategies. 



category 


Q's pure quantum strategies 


Picard's mixed quantum strategies 


qty 


1 


Ul=U(f ,*), U2=U(*,§ ) 


mixed a\ and & 3 


43 


2 


Ul=U(f *), U2=U(* § ) 


mixed 02 and / 


53 


3 


U1=U(0 *), U2=H 


mixed 03 and 02 


3 


4 


Ul=U(f ,*), U2=U(f ,0) 


mixed / and u\ 


1 



Table 3: Descriptions of the evolved quantum strategies. 



strategy 


unitary matrix 


0*1 


tf(f,7T) = 


1 

1 






t/(f,o) = 


-1 

1 




C"3 


U(0,n) = 


1 
-1 




I 


1/(0,0) = 


1 
1 




Hadamard(H) 




1 1 

1 -lj 
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For each of the 4 categories, we analyze the evolved quantum strategies 
in the following subsections. 



5.2.1 Category 1 Quantum Strategies: 
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Figure 7: Evolved Q strategies Figure 8: Evolved Picard strategies. 



Among the 100 simulation runs, 43 of them converged to strategies in 
category 1, where Q used U(^,*) and U(*, |) to play against Picard's 
mixed o~\ and 03 strategies. As analyzed in simulation 1, <f> of Ul does 
not have impact on p\. Q could therefore apply any U(^,*) to transform 



the initial state po 



tO Pl = U(l,: 



Next, Picard applied mixed o\ and <r 3 to transform the penny to p2 



{pro)a 1 pia[ + (l-pro)a 3 pia 



1 

2pro ■ 



2pro - 
1 



1 



Finally, Q applied 



U(*,|) and transformed the penny to ps 



C/(*,f)p 2 C7(*,f)t 



When measured, the penny collapses to |0) and to 1 1) with equal probability, 
hence both players received expected payoff 0. 

Note that 9 of Q's U2 has no impact on the final state P3, hence the mea- 
surement of the penny. This is because given an arbitrary strategy U(6, (p) = 



cost) — e 
sinO e l 
1 

2pro — 1 
1 a 
a 1 



'''sine 
'cos9 
2pro - 
1 



the state p% 
1 



cosv 
e~ l ^sin9 



U(8,(f>)p 2 U(9,^ 
sinO 



b cos9 



cosO —e^sind 1 
sinO e^cosO 2 
sinO 2 + cosO 2 a 

a sinO 2 + cost 



Since all U( 



# 7T N 



, 2 [ ) produce the p% that give the same measure- 
ment result, 9 of U2 can be any value between and J. In our simulation 
runs, 9 of U2 averaged to | (see Figure [7|). Similarly, Picard's pro also does 
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not have impact on p%. Figure [8] shows that Its values frustrated between 
and 1 and averaged to between 0.4 and 0.6. 



5.2.2 Category 2 Quantum Strategies: 
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Figure 9: Evolved Q strategies Figure 10: Evolved Picard strategies. 



53 of the 100 simulation runs converged to strategies in category 2, where 
Q used U(^,*) and U(*,|) to play against Picard's mixed 02 and /. With the 
[1 Ol 

, arter (4 applied u ( 4 . 
1 1 



initial state po 



after Q applied U(f , *), the state of the penny was 



Pi 



1 




1 1 



Next, Picard applied mixed 02 



1 a 
a 1 



When measured, the penny 



and I, which transformed the penny to P2 = (pro)o~2Pio~2 + (1 — pro) I pifl = 

. JP ro _ Finally, Q applied U(* 5) and transformed the 

1 — Ipro 1 - 

penny to p 3 = U(*, ^)p 2 U(*, §)t = 

collapsed to |0) and to 1 1) with equal probability, hence both players received 
expected payoff 0. 

Similar to the strategies in category 1, both <f> of Q's Ul and 6 of Q's 
U2 have no impact on the penny's final state p 3 . Figure [9] shows that their 
values have a wide range and averaged to ^ and ^. Similarly, Picard's 



pro has no impact on the penny's final state. Figure 10 shows its values 
fluctuated between and 1, with an average between 0.4 and 0.6. 
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Figure 11: Evolved Q strategies Figure 12: Evolved Picard strategies. 



5.2.3 Category 3 Quantum Strategies: 



3 of the 100 simulation runs converged to strategies in category 3, where Q 
uses U(0, *) and Hadamard to play against Picard's mixed 03 and 02 strate- 

[1 0" 




gies. With the initial state po 



, after Q applied U(0, *), the state 



of the penny remained the same: p\ = U (0, : 



[7(0, *)t 



Next, Picard applied mixed 03 and 02, which transformed the penny to 



Pi 



(pro)a 3 pial + (1 - pro)a 2 p\o 2 
Hadamard and transformed the penny to p% 



pro 








1 — pro 



Finally, Q applied 
2pro — 



1 

2pro — 1 1 

When measured, the penny collapsed to |0) and to |1) with equal probability, 
hence both players received expected payoff 0. 

Similar to the strategies in the two previous categories, of Q's Ul 
and pro of Picard's mixing probability have no impact on the penny's final 
state. As a result, they did not converge to a particular value. However, 
their standard errors are much lager than that of the strategies in the two 
previous categories, because they are averaged over 3 runs, instead of a 
larger number of simulation runs. 

5.2.4 Category 4 Quantum Strategies: 

Only 1 out of the 100 simulation runs converged to strategies in cate- 
gory 4, where Q used U(|,*) and U(f ,0) to play against Picard's mixed 



/ and <7i strategies. With the initial state po 



1 




after Q applied 
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Figure 13: Evolved Q strategies Figure 14: Evolved Picard strategies. 
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the state of the penny is p\ = U( 



2 ' ' 



1 





1 



Next, Picard applied mixed I and o\ strategies to transform the penny 



to p2 = (pro)Ip\I^ + (1 — pro)a\pia\ 



1 



Finally, Q ap- 



- pro 
pro 

plied U(5,0) which transformed the penny to p3 = U( ^, 0)p2^(f , 0)^ 
1 1 — 2pro 
1 — Ipro 1 

to 1 1) with equal probability, hence both players receive expected payoff 0. 
Similar to the strategies in the previous three categories, (j) of Q's Ul and 
pro of Picard's mixing probability have no impact on the penny's final state. 
Therefore, they did not converge to a particular value. 



4 i v 4 • 

When measured, the penny collapses to |0) and 



5.2.5 Discussion 

Categories 1 and 2 strategies are NE because neither of the two players can 
change his strategy alone to improve his payoff. However, this is not the 
case categories 3 and 4 strategies. For example, instead of using U(0,*) and 
H to force a tie, Q can use U(|, *) and U(f , 0) to beat Picard's mixed 
o"3 and <T2- However, if Q play that strategy, Picard can use the mixed I 
and 0i strategies to beat Q. Then if Picard use that strategy, Q has another 
winning strategy U(^,*) and H that can beat Picard and win the game. But 
if Q use that strategy, Picard can use mixed 03 and 02 to beat Q. This loops 
back to our starting point where Q has a winning strategy U(|,*) and U(^, 
0) to beat Picards mixed 03 and 02. 

This circular competition relationship among Q and Picard's winning 
strategies seems to suggest that there is no equilibrium strategies to settle 
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those winning strategies. However, under the competitive co-evolution of the 
EGT model, where both P and K populations are allowed to continuously 
evolve new strategies to play against the new strategies evolved by the other 
player, a new set of compromised strategies emerged. The strategies in cat- 
egories 3 and 4 are able to play against the other player's winning strategies 
and force a tie. Together, the four categories of quantum strategies form 
the ES set of this version of the quantum penny flip game. 

Compared to simulation 1, where Picard's mixed-two classical strategies 
in the ES set were not able to change the state of the penny transformed by 
Q's quantum strategy, hence lost the game every time, in this simulation, 
the mixed-two quantum strategies in the ES set allowed Picard to always 
force a tie. In other words, quantum strategies have benefited Picard in this 
version of the quantum penny flip game. 

6 Concluding Remarks 

Classical game theory is a mature science that is frequently applied to an- 
alyze conflicts that arise during decision making in economics and social 
sciences. With the recent development of quantum information processing, 
many classical games have been quantized to investigate the game dynamics 
under the influence of quantum mechanics. However, there has not been 
work applying evolutionary game theory (EGT) models to investigate quan- 
tum game theory. This work proposed and developed an EGT model to 
investigate the quantum penny flip games. In particular, we used the model 
to conduct a series of simulations where a population of mixed classical 
strategies from the ES set of the game were invaded by quantum strategies. 
The results of our investigation are very encouraging. 

First, we found that when only one of the two players' mixed classical 
strategies were invaded, the results were different. In one case, due to the in- 
terference phenomenon of superposition, quantum strategies provided more 
payoff, hence successfully replaced the mixed classical strategies in the ES 
set. In the other case, the mixed classical strategies were able to sustain the 
invasion of quantum strategies and remained in the ES set. Secondly, when 
both players' mixed classical strategies were invaded by quantum strategies, 
a new quantum ES set emerged. The strategies in the quantum ES set give 
both players payoff 0, which is the same as the payoff of the strategies in 
the mixed classical ES set of this game. 

With the established EGT framework, we will continue our investigation 
of mixed quantum ES set in the quantum penny flip game. In particular, 
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we will increase the number of quantum strategies used by each player to 
identify other quantum ES sets in this game [TT]. We are also interested in 
applying the developed methodology to study other quantum games. 
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